DropIn: Making Reservoir Computing Neural Networks Robust to Missing Inputs by Dropout

摘要

The paper presents a novel, principled approach to train recurrent neuralnetworks from the Reservoir Computing family that are robust to missing part ofthe input features at prediction time. By building on the ensembling propertiesof Dropout regularization, we propose a methodology, named DropIn, whichefficiently trains a neural model as a committee machine of subnetworks, eachcapable of predicting with a subset of the original input features. We discussthe application of the DropIn methodology in the context of Reservoir Computingmodels and targeting applications characterized by input sources that areunreliable or prone to be disconnected, such as in pervasive wireless sensornetworks and ambient intelligence. We provide an experimental assessment usingreal-world data from such application domains, showing how the Dropinmethodology allows to maintain predictive performances comparable to those of amodel without missing features, even when 20\%-50\% of the inputs are notavailable.